AIAA-94-1 286-CP 


Grasping Objects Autonomously in Simulated KC-135 Zero-G 


Robert S. Nors worthy 
Intelligent Systems Department 
Lockheed Engineering & Sciences Co. 
Houston, Tx 77058 
robertn@superman.jsc.nasa.gov 


ABSTRACT 

The EVAHR (Extravehicular Activity Helper/Retriever) 
robot is being developed to perform a variety of 
navigation and manipulation tasks under astronaut 
supervision. The EVAHR is equipped with a manipulator 
and dexterous end-effector for capture and a laser range 
imager with pan/tilt for target perception. Perception 
software has been developed to perform target pose 
estimation, tracking, and motion estimation for rigid, 
freely rotating, polyhedral objects. Manipulator grasp 
planning and trajectory control software has also been 
developed to grasp targets while avoiding collisions. 

Right experiments have been scheduled aboard NASA’s 
Reduced Gravity Laboratory (KC-135 aircraft) in 1994 to 
attempt grasps of free-floating objects. A software 
simulation of the EVAHR hardware, KC-135 flight 
dynamics, collision detection, and grasp impact dynamics 
has been developed to integrate and test the EVAHR 
software. This paper describes the EVAHR system, with 
emphasis on the robotic and KC-135 simulation software. 

1. INTRODUCTION 

Much work has been done on autonomous grasping of 
stationary, complex objects, e.g. in bin-picking and pick- 
and-place problems! 1,2]. However, very little work has 
been devoted to autonomous grasping of moving, 
complex objects. This is probably due in part to the 
difficulty of the problem and to a lack of applications for 
such technology. There exists an abundance of 
applications for this technology at NASA, however, as 
recent Shuttle-based satellite recovery attempts have 
amply demonstrated. STS-49 required 3 astronauts to 
grasp the ailing INTELSAT VI satellite - an autonomous 
mechanism would have spared the astronauts hazardous 
EVA activity. The difficulties encountered on 41-C in 
1984 rescuing the Solar Max satellite and the failure to 
recover the Leasat 3 during 5 ID in 1985 provide further 
evidence of such a need. The recent ROTEX experiment 
onboard the Shuttle mission STS-55[3] demonstrates that 
the German Aerospace Research Establishment (DLR) 
considers this a key piece of robotic technology for space. 
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The ROTEX system combined a tele-operated manipulator 
and camera system to grasp a free-floating object, among 
other tasks. 

The EVAHR (Extra-Vehicular Activity Helper/Retriever) 
project has been developing robotic technology, hardware 
and software, for navigating among and grasping free- 
floating objects since 1987. The EVAHR project was 
originally conceived as an autonomous device for 
retrieving personnel or objects which had become detached 
from Space Station Freedom (SSF). At the conclusion of 
Phase II in 1990, the following autonomous robotic 
capabilities were demonstrated on a precision air bearing 
floor (PABF): a) navigation among stationary obstacles 
and b) grasping arbitrarily located and oriented, cylindrical 
targets. At this time, a simulation of the EVAHR in an 6 
degree-of-freedom (DOF) orbital environment[4] was 
developed for use in the next phase. It would provide a 
testbed for development of 6 DOF navigation and grasping 
software. Models initially developed included orbital 
dynamics, plume impingement effects of the Manned 
Maneuvering Unit (MMU) (a compressed-gas-based 
propulsion system), and a laser scanner range/intensity 
image simulation[5]. Six DOF body motion control and 
orbital state estimation algorithms were developed and 
tested for navigation purposes. Phase III then took as its 
primary goal the development and demonstration of an 
integrated system for grasping free-floating, complex 
objects. 

The EVAHR team chose the KC-135 zero-gravity aircraft 
as the hardware testing and demonstration platform for this 
phase[6]. The KC-135 aircraft flies a series of parabolas, 
free-falling over the hump of each parabola, simulating 
zero-gravity or weightlessness for objects and personnel 
free to float inside the plane. Each of these periods lasts 
20-25 seconds. The EVAHR, attached to the floor of the 
KC-135 throughout the experiments, will attempt to 
grasp targets released in its envelope during the simulated 
zero-gravity period . While the KC-135 is a difficult 
environment for such testing, it provides the only direct 
means of testing the contact dynamics of free-space 
grasping short of a Shuttle flight experiment. 

Prior to flight, software to grasp free-floating, complex 
objects in a 6 DOF, orbital environment was developed 
and integrated. Software for an orbital environment was 
developed first and testing was conducted to determine the 
performance of the system[7]. Dynamics and state 
estimation software was then developed to represent the 
KC-135 environment. 
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2. OBJECTIVES 

Essentially, the integrated EVAHR grasping software 
must be capable of autonomously grasping a 
variety of polyhedral objects, each in less than 
15 seconds, where the object is freely but 
slowly translating and freely rotating at up to 
30 deg/sec. 

A maximum time-to-grasp of 15 seconds was initially 
chosen as a conservative estimate of the zero-gravity 
period provided by the KC-135. While recent data 
gathering flights on-board the KC-135 have shown that 
the time available is actually less than 8 seconds, most of 
the testing presented here was conducted with the 15 
second per grasp attempt time limit. One might speculate 
that in order to be considered useful in real applications, 
the robot would have to perform these grasps much more 
quickly, but this consideration did not affect our 
requirements. 

The maximum rotational rate is taken from a Crew and 
Equipment Retrieval Study (CERS) conducted by 
NASA[8], which estimated the likely maximum 
separation rates of objects which might become detached 
or untethered from the SSF. The CERS maximum 
translational rate of 2 feet/sec was ignored because the 
EVAHR, as a free-flying vehicle, would rendezvous with 
and stationkeep at the target with nearly the same 
velocity. From on-orbit navigation tests in the EVAHR 
simulation, a rate of 2 in/sec during stationkeeping was 
found to be the maximum. However, it would be 
impractical for the EVAHR vehicle to match the CERS 
rotational rates. 

To enable the EVAHR to meet these objectives, a 
metrically-accurate CAD surface model and mass 
properties (mass, center-of-mass, inertia matrix) are 
provided to the EVAHR describing each target object prior 
to runtime. The target's surface must be non-specular so 
that the laser scanner can image it. 

Integrated software testing with the orbital simulation 
yielded information about the performance of the EVAHR 
software in its ultimate target environment, i.e. the 
success ratio for different target velocities, the required 
vision update rates, etc. The addition of a KC-135 
dynamics model provided a means of integrating the KC- 
135 state estimation module and verifying continued, 
successful system performance. 

3. EVAHR HARDWARE 

The principal EVAHR hardware elements to be flown on 
the KC-135 flight experiments is shown in Figures 1 and 
2. These elements are combined into a fictitious EVAHR 
body in the orbital simulation shown in Figure 3. A 
Perceptron LASAR laser scanner is mounted in a pan/tilt 
mechanism on top of the EVAHR body. The Perceptron 
provides 64x256 range/reflectance images at 9 Hz, 


128x256 resolution at 5 Hz, or 256x256 at 2.5 Hz - 
switchable at runtime. The range resolution is 
approximately 1/7 inch, though noise can result in a range 
error of up to 3%. The pan/tilt motors are able to rotate 
the Perceptron at 180 deg/sec. A Robotics Research (RR) 
K807i 7 DOF manipulator is mounted as a left arm, with 
a 3-fingered JH4 dexterous hand as its end-effector. The 
RR arm has a maximum end-effector speed of 30 in/sec, 
static repeatability of .002 in, and may accurately support 
loads of 10 lbs. The dexterous hand was manufactured at 
JSC. It provides 3 joints per finger, though only 2 are 
active. The maximum finger joint speed is roughly 120 
deg/sec so that the hand is able to close in roughly 1/4 
sec. Proximity detectors are located on the palm and each 
finger tip. 



Figure 1. EVAHR laser scanner & pan/tilt 



Figure 2. EVAHR manipulator/hand 
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When the EVAHR is flown on the KC-135, the EVAHR 
hardware will be fixed to the floor of the aircraft - only the 
target will float free. In the orbital configuration, a 24- 
thruster Manned Maneuvering Unit (MMU) [9] is strapped 
to its back to provide 6 DOF propulsion. The MMU 
model was turned off during the grasping tests. Not shown 
in the figures is an inertial measurement unit (IMU) 
composed of 3-axis accelerometers and rate gyros, which 
provide acceleration and rotational velocity measurements 
of the EVAHR body. The EVAHR will use the following 
computers to grasp free-floating targets in real-time 
onboard the KC-135: 8 Intel i860 processors connected via 
crossbar, 14 Transputer T800 processors networked via 
synchronous Transputer links, and 4 68040 processors. A 
VMEbus will connect the processors/ networks and 
sensor/actuator electronics. 



Figures 3. EVAHR grasping ORU in orbital simulation. 


4, EVAHR SIMULATION 

Figure 3 shows testing with the orbital simulation. The 
arm and laser scanner will be attached to the floor of the 
KC-135 in flight, as in Figures 1 and 2, rather than to the 
body. The KC-135 simulation includes models of the a) 
EVAHR hardware and b) dynamics of the KC-135 
environment. Models of the EVAHR hardware required to 
test autonomous vehicle navigation were developed 
initially. One of these models, the laser scanner image 
simulation software, continues to play a major role in 
vision-guided manipulation. This simulation takes 
advantage of the z-buffering graphics hardware associated 
with Silicon Graphics workstations to provide fast range 
image generation for polyhedral-surface objects. The 
model was extended to provide configurations possible in 
the Perceptron laser scanner - 64x256 or 256x256 images 
and variable field of view. The laser scanner simulation 
model does not try to model noise or distortions, however, 
other than uniform pixel noise. Preliminary tests with the 
actual Perceptron hardware show a maximum error of 3% 


in the range pixel measurements. Unfortunately, the 
Perceptron produces more systematic noise which is not 
modeled by this simulation. Target CAD models (see 
Figure 6) along with CAD models of the EVAHR 
hardware (see Figure 3), are used by the laser scanner 
simulation to generate simulated images. 

Additional models were developed expressly to support 
testing of the grasping software. These include models of: 
a) the RR manipulator and JH4 dexterous hand, b) 
collision detection, and c) grasp contact dynamics. Because 
dynamic models of manipulators are hard to obtain and 
compute-intensive, we opted for an acceleration-level 
model of the RR 7 DOF arm and JH-4 dexterous hand. 
This model has since been proven in tests with the actual 
arm. Collision detection software was developed to detect 
a) desired collisions between the JH-4 palm/fingers and the 
target and b) collisions that the manipulator trajectory 
control software is attempting to avoid. The collision 
detection software requires that the objects be convex or be 
composed of convex subparts. The dynamics of each body 
(e.g. EVAHR, target) is computed by summing the forces 
and moments on each orbiting body, computing the 
associated translational and rotational accelerations, then 
integrating (at 100 Hz) to arrive at each body's velocity 
and position. One such translational force is gravity. 
When collisions between the fingers or palm of the JH4 
hand and the target are detected, additional forces and 
moments on the target are computed and "folded into" the 
(orbital) dynamics state propagation. To make the 
computation tractable, each finger is assumed massless 
relative to the target and stops on contact: however, its 
motion is resumed if the forces/moments on the target are 
such that the target moves away from the finger. The 
forces/moments are computed one contact point at a time, 
even when multiple fingers are in contact simultaneously. 
Assuming a loss of velocity after each contact due to 
friction, the motion of the target is considered "stopped" 
when the target's total velocity falls under a threshold. 
This simulation has not been validated except by visual 
verification of the results of random test cases. Numerous 
cases have shown the target slip out of the grasp of the 
closing fingers as well as the target being trapped by them 
- the behavior has appeared "reasonable". 

5. EVAHR SOFTWARE 

The EVAHR grasp software acts as a closed-loop control 
system (see Figure 4), where the simulation models just 
described act as the plant. The vision and grasping 
modules are as described in 7. All are implemented in C, 
as are the simulation models. 

Measurements of the target's pose (3D position/3D 
orientation) are computed by the vision modules 
Tracking[10] and Pose Estimation! 11, 12]. The Tracker 
processes each image to find a rough estimate of the 
target's location and, if the pose estimator is ready, the 
contour of the object. To avoid confusing the target with 
the arm/hand, which may be in the image or even partially 
obscuring the target, the arm/hand is removed from each 
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image. The is accomplished by using a CAD model of the 
arm/hand. Pose estimates are computed by finding the best 
match of image features (edges/vertices) on the occluding 
contour to corresponding features in CAD model 
representations of each target (see Figure 6). The pose 
estimation software uses the same CAD models as the 
laser scanner simulation uses to draw its images and thus 
benefits from perfect surface geometry models of the 
targets. These measurements are computed at rates which 
vary depending upon a number of factors: the 
complexity/symmetry of the object, the availability of 
predicted poses for the image time, the degree of occlusion 
by the EVAHR arm/hand, and of course the computing 
resources available. The output rate of the pose estimator 
may vary from .1 sec (laser scanner minimum image 
interval) to several seconds. 

These measurements are provided to independent Kalman 
filters which estimate the target's translational and 
rotational states. These model-based state estimators are 
different from those in the orbital EVAHR software, due 
to the different environment dynamics, and are described 
more fully in [13]. The rotational filter required only 
minor modifications, namely redefinition of the inertial 
coordinate system (CS). While the orbital version used an 
Earth-centered CS, the KC-135 version uses the aircraft's 
orientation at the beginning of each parabola. Thus the 
target’s attitude is EVAHR-relative from that point on, 
due to the arm being fixed to the aircraft. The KC-135 
translational state estimator had to be made explicitly 
EVAHR-relative, due to the lack of the Earth-based 
position correction (e.g. Global Positioning System 
(GPS)) that was assumed in the orbital software. The 
target's state is thus estimated in the EVAHR's coordinate 
system, using inertial measurements from the IMU and 
initializations/corrections from vision. For the purposes 
of this round of simulation testing, both the 
measurements and corrections were taken, noiseless, 
directly from the dynamics simulation. Both the 
translational state (position, velocity, acceleration) and 
rotational state(attitude, rotational velocity and 
acceleration) states are computed at 100 Hz for the 
EVAHR Manipulator Trajectory Controller. They are also 
archived in the World Model database as predictive 
feedback to the Tracking/Pose Estimation modules for 
images to be processed shortly thereafter. 

The Manipulator Grasp Planning and Trajectory Control 
module[14] receives an estimated target state at 100 Hz. 
At 10 Hz, the Grasp Planner decides which of the multiple 
grasp regions available for the target is optimal for 
grasping. (Prior to runtime, the graspable regions on the 
target's surface were computed and placed in a database. 
These regions were found automatically based on CAD 
models of the JH4 hand and the target.) The Manipulator 
Trajectory Controller (MTC) computes RR/JH4 joint 
accelerations to track the motion of the chosen grasp 
region on the target at 100 Hz. Each cycle the MTC also 
computes joint accelerations to avoid a) joint 
singularities, b) joint limits, c) RR link-link collisions, 
d) RR-EVAHR body collisions, and e) collisions between 
the RR/JH4 and regions on the target which are not to be 


grasped. The MTC uses a potential field-based algorithm 
to compute these avoidance accelerations. 
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The MTC takes advantage of the extra DOF in the 
manipulator to minimize the effects on the end-effector 
trajectory. 

The cycle rate of 100 Hz was determined primarily by the 
need to control RR joint accelerations at that rate. It was 
also determined previously that propagation of the target's 
rotational state would diverge if the integration timestep 
were less than approximately 50 Hz. 

The software was run on 2 Sparcstations and 1 Silicon 
Graphics 70GT for the experimental testing. Figure 5 
shows a representative sample event timeline for an 
attempted grasp of the NBox. 

Two different types of target shapes were used in testing: 
spherical and polyhedral. In the case of the sphere, the 
pose and rotational state estimation modules are 
unnecessary and grasping is greatly simplified as well. 
Therefore, polyhedral objects were used almost exclusively 
for testing. Three different polyhedral shapes were used 
(Figure 6). 



Jettison handle (JettH) Orbital replacement unit (ORU) 


NBox Sphere 

Figure 6. Polyhedral and spherical target shapes 

An additional major difference between the oibital and KC- 
135 software lies in the area of image segmentation. 

While the background of space provides an extremely 
convenient basis for segmenting the target in each image, 
the inside of the KC-135 aircraft is very difficult and 
compute-intensive. To minimize the computational load, 
a black curtain will cover the inside of the aircraft to 
simulate space. 

7. CO NC LUSION 

The EVAHR software for grasping a target floating free 
on-orbit has been described, as well as the KC-135 


simulation for testing the software as an integrated whole. 
The software was integrated and successful grasps were 
achieved, similar to those obtained in thorough, integrated 
testing with the orbital simulation. This result suggests 
that the software is ready for flight testing on-board the 
KC-135. 

8. FUTURE WORK 

A preliminary KC-135 flight test using spherical targets 
exclusively will be conducted in the near future. The 
advantage of spherical targets is that they have no 
rotational state and, hence, neither the pose estimation 
software nor the rotational Kalman filter are required. A 
Teleos PRISM3 stereo-vision system, instead of the 
Perception laser scanner, will be used to provide target 
position inputs (at > 20 Hz). The translational state 
estimation and grasping software modules have been 
reimplemented on parallel Intel i860 and Siemens 
Transputer architectures to achieve real-time speeds. 
Calibration of the arm and PRISM3 systems has been 
completed and pre-flight integrated systems testing is to 
begin shortly. 

Meanwhile, modifications are being made by Perceptron 
to the laser scanner to improve its noise characteristics. Of 
primary concern is range drift, which is serious enough to 
prevent the sensor from being calibrated. Assuming that 
the problems with the laser scanner are corrected, a flight 
with the polyhedral targets is scheduled for sometime in 
the summer of 1994. 
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